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We discuss recent progress in the development of cognitive ontologies and summarize 
three challenges in the coordinated development and application of these resources. 
Challenge 1 is to adopt a standardized definition for cognitive processes. We describe three 
possibilities and recommend one that is consistent with the standard view in cognitive and 
biomedical sciences. Challenge 2 is harmonization. Gaps and conflicts in representation 
must be resolved so that these resources can be combined for mark-up and interpretation 
of multi-modal data. Finally, Challenge 3 is to test the utility of these resources for large- 
scale annotation of data, search and query, and knowledge discovery and integration. As 
term definitions are tested and revised, harmonization should enable coordinated updates 
across ontologies. However, the true test of these definitions will be in their community- 
wide adoption which will test whether they support valid inferences about psychological 
and neuroscientific data. 

Keywords: ontology, cognition, mental functioning, neuroscience, annotation, integration, big data, brain science 



INTRODUCTION 

Mental processes unfold at multiple levels, from the cellular 
dynamics of memory consolidation to sensory and motor behav- 
iors that engage the entire brain and body. Thus, a comprehen- 
sive understanding of the phenomena associated with cognition 
requires coordinated contributions from at least psychology, biol- 
ogy, and neuroscience (Albus etal., 2007). The last two decades 
have seen unprecedented steps toward this goal, with accelerated 
use of neuroimaging (PET and fMRI), neurophysiology (EEG), 
and neurogenomics, together with experimental paradigms from 
cognitive, clinical, and neuro-psychology (Jones and Mendell, 
1999). 

While these methods have produced a wealth of high-quality 
data about mental functioning 1 , major advances will require not 
just more data, but new ways of integrating, processing, and under- 
standing the data (Decety and Cacioppo, 2010; Akil etal., 2011). 
Recently, ontologies have emerged as a key tool to support these 
efforts. An ontology is a formal representation of the types of 
entities in a given domain, along with their properties and inter- 
relationships. Recent projects, such as the Cognitive Paradigm 



1 For the purposes of this article, the authors agree to use "mental process," "mental 
functioning," and "cognition" synonymously. Distinctions are often made between 
these, and while there can be good reason to do so in certain cases, clarifying the 
distinctions here would go beyond our scope. 



Ontology (CogPO, Turner and Laird, 2012), Neural Electromag- 
netic Ontologies (NEMO, Frishkoff etal, 2009, 2011a,b), the 
Neuroscience Information Framework (NIF, Gardner et al., 2008; 
Imam etal., 2012), and the Cognitive Atlas (Poldrack etal, 2011), 
show that ontologies can be used to link data across neuroscien- 
tific databases and other resources (e.g., Imam etal., 2012) and 
to facilitate meta-analyses of neuroimaging data (e.g., Laird et al., 
2009; Frishkoff etal., 2011a; Fox and Friston, 2012; Turner and 
Laird, 2012). 

The present paper summarizes these recent efforts and 
describes three challenges in the development and integration 
of cognitive ontologies. Challenge 1 is to adopt standardized 
definitions for cognitive processes, and integrate cognitive pro- 
cesses within the framework offered by a foundation (upper- 
level) ontology shared with other biomedical ontologies. We 
describe three possibilities and recommend one that is con- 
sistent with the standard view in cognitive and biomedical 
sciences. 

Challenge 2 is harmonization of terms across ontologies. A 
search for "memory" in BioPortal (Noy etal., 2009) returns over 
150 terms from 30 ontologies 2 . These and other basic terms, 
such as "perception," "planning," and "emotion," have multiple 
and sometimes conflicting definitions across ontologies. These 



2 http://bioportal.bioontology.org/search?query— memory&commit— Search 
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conflicts must be resolved so that multiple ontologies can be com- 
bined for mark-up and interpretation of multi-modal data. In 
the present paper, we describe the scope of several cognitive and 
biomedical ontologies and suggest areas for improved coordina- 
tion in their development and application. 

Challenge 3 is to test the utility of cognitive ontologies for 
large-scale data annotation, search and query, and ontology- 
based analysis. While Challenge 1 is concerned with ontology 
development, Challenges 2 and 3 reflect an interest in ontology 
harmonization and application. As term definitions are tested 
and revised, harmonization should enable coordinated updates 
across ontologies. However, the true test of these definitions 
will be whether they support valid inferences about psycho- 
logical and neuroscientific data. We conclude with our view 
on the future of the development and application of cognitive 
ontologies. 

CHALLENGE #1: DEFINING AND CLASSIFYING MENTAL 
PROCESSES 

Understanding the relationship between mental functioning and 
brain activity is a unifying aim for biology, neuroscience, psy- 
chology, and related fields. Upper-level ontologies, such as the 
Basic Formal Ontology (BFO; Smith and Grenon, 2004), can 
provide domain-independent integrating classes and domain- 
bridging relationships that can serve as a starting point for the 
annotation of data arising from this difficult search for under- 
standing. Anatomical entities such as the central nervous system 
are classified as objects within BFO, and BFO also allows for "fiat" 
object parts for the classification of entities that have been arbitrar- 
ily (but usefully) divided into sections, e.g., the nervous system is 
divided into the central and peripheral nervous systems, although 
they are physically continuous through the connections of neurons 
residing in each division. Brain activity is classified as a process in 
BFO, and the anatomical entities are related via a participation 
relation to those processes. 

Within BFO there are three possible ways to represent the rela- 
tionship between mental processes and brain processes (Frishkoff, 
2012). According to the first view, mental processes are distinct 
from physical (e.g., brain) processes. This is a folk psychological 
view and represents a kind of dualism: there are physical enti- 
ties, and there are nonphysical (mental) entities. In contrast, the 
second and third views represent mental processes as a subclass 
of physiological (bodily) processes. The second view characterizes 
mental processes as a type of bodily process that is distinct from 
brain processes, while the third view holds that mental processes 
are a subtype of brain processes. 

The first view, dualism, may generate hypotheses that are dif- 
ficult or impossible to test within a scientific framework and is 
justifiably scientifically unpopular. In contrast, the second and 
third views are consistent with what we know about the phys- 
iological basis of cognition. Both assert that mental processes 
are physical processes, while the third view goes one step fur- 
ther in asserting that mental processes unfold solely in the brain, 
not the autonomic (reproductive, digestive) system or periph- 
eral sensory and motor areas (e.g., hands, feet). This view is 
consistent with much of what we know about sensorimotor func- 
tions, emotion and motivation, and more complex cognitive 



processes. On the other hand, if we wish to embrace a more 
embodied view of cognition, we could argue that interactions 
between the brain and body (visceral and somatic processes) are 
integral to cognition. Therefore, we may not wish to exclude 
physiological processes outside the brain in representing mental 
phenomena. For this reason, we recommend either the sec- 
ond or third view as a reasonable starting point for cognitive 
ontologies. 

Like any complex system, cognition comprises multiple sub- 
systems. Most researchers would agree that cognitive subsystems 
include at least perception, attention, short- and long-term 
memory, decision-making, language, and emotion (James, 1890; 
Poldrack etal., 2011). Consequently, any ontology of mental 
functioning is likely to include terms for at least these sorts of phe- 
nomena. However, many finer-grained distinctions can be drawn. 
Theories of cognition differ in whether and how they draw these 
distinctions, and in how the subsystems assumed by the theory are 
thought to interact and give rise to patterns of behavior. 

Mental phenomena cannot be observed in the same way that 
behavior or brain activity can be observed. A scientific study must 
therefore "operationalize," or define, a mental process with respect 
to a particular experimental framework. To this end, researchers 
have devised a variety of experimental paradigms (Turner and 
Laird, 2012). Each paradigm includes a set of explicit instructions 
for subjects to behave in certain ways (e.g., attend, then push a 
button) in response to different stimuli (e.g., words or faces). Out- 
come measures include response time and accuracy, as well as 
patterns of physiological (bodily or brain) activity. For example, 
Fliessbach etal. (2010) examined how the strength of a mem- 
ory (operationalized in terms of response accuracy) depended on 
the way that people learn, or encode, new information. Some 
participants were asked to perform an orthographic (letter) judg- 
ment, while others were asked to perform a semantic (meaning) 
judgment. Results showed that memory recall was better for the 
semantic task, providing support for a familiar construct in psy- 
chology of memory, known as "depth of processing." This study 
illustrates how experimental variables are used to operationally 
define a mental phenomenon. At the same time, we note that 
operational definitions rely on particular measurement meth- 
ods, and these methods may not be sensitive to all aspects of 
the phenomenon of interest, or may reflect additional processes, 
e.g., so-called task demands. Therefore, converging evidence from 
multiple methods is essential for studies of cognition. 

In this context, the BrainMap database (Laird etal., 2005) and 
the Cognitive Paradigm ontology (Figure 1, Turner and Laird, 
2012) have been used to annotate behavioral and brain data 
in order to test hypotheses about brain-behavior relationships. 
BrainMap is a large curated repository of cognitive neuroimaging 
studies that can be used to explore relationships between behav- 
ior and patterns of brain activity (mainly from PET and fMRI). 
It encodes core experiment metadata, including the conditions, 
tasks, responses and measurement methods. Using this struc- 
ture, experimental results can be grouped together, based on the 
objective similarities among the stimuli, task instructions, and 
measures, regardless of whether the experiment was designed to 
study a particular subsystem, such as memory, executive function, 
or attention (Burns and Turner, 2013). 
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FIGURE 1 | A partial representation of the Cognitive Paradigm Ontology (CogPO). 



The Cognitive Atlas (Poldrack etal., 2011) represents a com- 
plementary effort. Using a wikipedia-style (or "bazaar" approach; 
Raymond, 1999), the Cognitive Atlas captures a wide range of 
terminology used in cognitive science, relating operational mea- 
sures to the types of mental functioning that they are designed 
to measure. For example, the Cognitive Atlas contains state- 
ments such as '"pattern comparison task' 3 measures 'processing 
capacity'" 4 . 

In clinical neuropsychology, behavioral and self-report mea- 
sures (called "neuropsychological tests") are widely used for 
the diagnosis and treatment of patients with psychological and 
developmental disorders. Neurological assessment rests on the 
theory that the nature and location of brain damage, either 
by disease or injury, can be discovered by testing patients' 
cognitive functioning. The Neuropsychological Testing Ontol- 
ogy (NPT 5 ) has been used to characterize these tests, which 
include measures of cognitive abilities, such as short-term vs. 
long-term memory recall or executive functioning. The Neu- 
rological Disease Ontology (ND 6 ) classifies entities relevant to 
neurological diseases, such as Alzheimer's and multiple scle- 
rosis. Diseases are classified and related to diagnostic criteria 
and to neuropathology. A key challenge in clinical and cogni- 
tive neuroscience is to understand how physiological disorders 
due to trauma or disease are related to cognitive deficits, such 
as memory loss or aphasia. We expect that harmonization 



3 http://www.cognitiveatlas.org/task/id/trm_4da86cb034ff6 
4 http://www.cognitiveatlas.org/concept/id/trm_4a3fd79d0adf4 
5 http://code.google.com/p/neuropsychological-testing-ontology/ 
6 http://code.google.com/p/neurological-disease-ontology/ 



of the NPT, ND, and similar resources, will support this 
goal. 

CHALLENGE #2: INTEROPERABILITY OF COGNITIVE 
ONTOLOGIES 

Several cognitive ontologies have been mentioned thus far. In 
recent years, psychology has embraced methods and insights from 
the biological sciences, including molecular genetics, evolution- 
ary biology, and physiology (Kandel, 2008). These methods show 
the brain at multiple orders of spatial resolution, from genes to 
neural maps to interacting systems. Likewise, medical research 
has increasingly turned to biological evidence for explanations of 
mental disorders, such as depression and schizophrenia (Weyandt, 

2006) . As a result, many bio-ontologies now also include termi- 
nology for mental processes. For example, the Gene Ontology 
(GO), widely used for cross-species annotation of the functions of 
gene products (Ashburner et al., 2000), includes terms for "cogni- 
tion," "learning," and "memory," among others. This inter-domain 
diversity of ontologies including terminology for cognitive entities 
is accompanied by diversity in definitions and classification. 

To coordinate work across domains and promote convergence, 
the Open Biomedical Ontologies (OBO) Foundry (Smith etal., 

2007) provides recommendations on best practices in ontol- 
ogy development, such as re-use of existing terms, community 
involvement (i.e. being "open"), and standardization of meta-data 
practices. In our experience, these standards have contributed to 
the quality and interoperability of neuroscientific and cognitive 
ontologies. For example, these ontologies have largely adopted 
BFO as a common upper ontology, and there is extensive reuse of 
mid-level ontologies, such as OBI, IAO, OGMS, and UBERON, as 
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Table 1 I An overview of ontology efforts for cognitive processes and related domains. 



Name/RRID 



Location 



Status Citation 



Scope 



Upper Related 



Mental Functioning 
Ontology (MF)/ 
RRID:nlx_157305 

Cognitive Paradigm 
Ontology (CogPO)/ 
RRID:nlx_155537 

Neural Electromagnetic 
Ontologies (NEMO)/ 
RRID:nif-0000-10899 

Neuroscience Information 
Framework (NIFSTD)/ 
RRID:nlx_144512 

Cognitive Atlas (COGAT)/ 
RRID:nif-0000-24591 



Neurological Disease 
Ontology (ND)/ 
RRID:nlx_157304 

Neuro Behavioral 
Ontology (NBO)/ 
RRID:nlx_151745 

Neuropsychological 
Testing Ontology (NPT)/ 
RRID:nlx_157303 



http://purl.bioontology. Alpha 
org/ontology/MF 

http://purl.bioontology. Release 

org/ontology/COGPO (v1.0) 

http://purl.bioontology. Release 

org/ontology/NEMO (v3.2) 

http://purl.bioontology. Release 

org/ontology/NIFSTD (v2.9) 

http://purl.bioontology.org/ Release 

ontology/COGAT (v1.0) 

http://code.google.eom/p/ Alpha 
neurological-disease-ontology/ 

http://purl.bioontology.org/ Beta 
ontology/NBO 

https://code.google.eom/p/ Alpha 

neuropsychological-testing- 

ontology/ 



Hastings etal. Cognitive processes BFO 2.0 

(2012) and dispositions 

Turner and Cognitive experimental BFO 1.1 

Laird (2012) paradigms 



Frishkoff etal. Cognitive and brain 



(2009) 

Imam etal. 
(2012) 



processes, EEG and 
ERP measures 

Brain anatomy 
(cross-species) 



BFO 1.1 



BFO 1.1 



Poldrack etal. Cognitive processes, - 

(2011) cognitive experimental 
paradigms 

Jensen etal. Neurological disorders, BFO 2.0 

(2013) neuropathology 

Gkoutosetal. Behavioral processes 

(2012) and phenotypes 



Cox etal. Cognitive assessment, BFO 2.0 

(2013) behavioral measures 



OBI/IAO, OGMS 



OBI/IAO, 
BrainMap 

OBI/IAO, NIFSTD, 
CogPO, UO 

OBI/IAO, 

UBERON, CogPO 
CogPO 



OBI, GO, OGMS, 
CL, ChEBI, PR 

GO, ChEBI, 
UBERON 

OBI/IAO 



OBI, Ontology of Biological Investigations; IAO, Information Artifact Ontology; OGMS, Ontology for General Medical Science; UO, Units Ontology; UBERON, Uber 
Anatomy Ontology; CL, Cell Ontology; ChEBI, Chemical Entities of Biological Interest Ontology; PR, Protein Ontology. 
All ontologies are available via http://www.obofoundry.org/ 



shown in Table 1. This re-use ensures that corresponding terms 
across ontologies share exactly the same identifier and definition, 
thus seamlessly harmonizing the resulting data annotations. 

A guiding principle of the OBO Foundry is that of ontological 
realism, which suggests that ontologies should aim "to identify the 
sorts of entities that exist... according to the best current scientific 
understanding" (Smith and Ceusters, 2010). Ontological realism 
is a foundation for BFO and, in our view, has effectively promoted 
a community-wide focus on representing scientific knowledge. 
Interestingly, ontological realism might be seen to be challenged 
by the need to represent certain mental phenomena, such as 
hallucination, that can reference entities that do not exist. One 
solution is to distinguish these processes, which can be viewed 
as non-canonical, from canonical perception, in which perceptual 
representations are triggered by sensory stimuli. Perception is rela- 
tional in the sense that it brings the subject into contact with an 
external object in the world (Mulligan and Smith, 1986). Just as the 



Foundational Model of Anatomy (FMA, Rosse and Mejino, 2003) 
contains terms for arm and leg, but not for amputation stump 
or shriveled arm, ontologies that represent canonical mental pro- 
cesses would contain terms for visual and aural perception, but 
not for synaesthesia or tinnitus. Terms like these would belong in 
extension ontologies relating to specific types of mental or neu- 
rological disease (Ceusters and Smith, 2010). By focusing on the 
canonical case, perceptual processes (for example) can safely be 
defined as representing objects and involving beliefs about those 
objects. A next challenge, then, is to capture the relationships 
between canonical and non-canonical processes, acknowledging 
that some core sub-processes in normal perception are similar, or 
even identical, to those in (e.g.) hallucinating. 

CHALLENGE #3: APPLICATIONS OF COGNITIVE ONTOLOGIES 

In addition to data annotation and integration, ontologies 
can be used for improved search and retrieval of other 
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resources and for reasoning over data within as well as across 
experiments. 

INFORMATION RETRIEVAL 

The Neuroscience Information Framework 7 (NIF) provides for 
search and retrieval across a wide range of resources, includ- 
ing publications, databases and other information sources. 
The search engine is backed by the NIF ontology (NIFSTD), 
which supports better formalization and integration of exist- 
ing resources. For example, the ontologies include semantically 
based mappings between related terms, such as "cognition" 
and "mental process." Therefore, an ontology-based search 
that includes only one of these terms still provides excellent 
coverage. 

DATA ANNOTATION 

Ontologies are increasingly used for data annotation. For example, 
the behaviors of model organisms are widely used as proxies for the 
study of mental processes and disorders in humans (e.g. Cryan and 
Holmes, 2005; Wu and Luo, 2005; Mathur and Guo, 2010). Build- 
ing on the GO, the NeuroBehaviour Ontology (NBO, Gkoutos 
etal., 2012) represents behavior and is used for model organism 
data annotation. Formalization of the relationships between the 
behavior and assumed homologies in mental functioning repre- 
sented in ontologies would support better data integration and 
translation in this domain. 

ONTOLOGY-BASED ANALYSIS 

Finally, ontologies can be used to reason across data, e.g., for 
classification of complex patterns (as supported by NEMO, Bilder 
et al., 2009; Frishkoff et al., 2009, 201 la,b; Poldrack et al., 2012). 

To enable cross-domain applications, Hastings etal. (2012) 
have proposed a Mental Functioning Ontology (MF) which 
includes inter-ontology bridging modules. A bridging module 
assigns semantic relationships between terms in different ontolo- 
gies, such as identity, parthood, or realization. Bridging modules 
exist independently of each of the source ontologies but enable 
applications to safely draw on both of the ontologies, harnessing 
the knowledge-based relational links between them. Our vision is 
that MF will support harmonization of search and analysis tools 
across biochemical, biological, and medical data. In doing so, we 
wish to achieve stronger links between biological research ("bench 
sciences"), translational research, and medical records. Due to the 
explicit relationship between disorder categories and lower-level 
mental processes, we further anticipate that MF might help to 
further disambiguate data in diagnostic categories for mental dis- 
orders that show high levels of co-morbidity (Hastings and Schulz, 
2012). 

OUTLOOK: WHAT IS NEEDED FOR THE FUTURE? 

In the last few years, multiple ontologies have been created for 
representation, annotation, and reasoning about cognitive func- 
tioning, and for the representation of related behavior and brain 
data in different experimental contexts. Table 1 gives an overview 
of these efforts. At present, there is no one project that unites 
these efforts, although there is considerable cross-pollination, 



7 http ://www.neuinfo. org/ 



as reflected in the "Related" column, which shows mid-level 
ontologies that are shared across projects. 

For researchers, mapping between different ontologies can 
present challenges, as independent ontologies may reflect different 
assumptions, methodological practices and scientific viewpoints 
(as reflected, for example, in conflicting definitions). It is inef- 
ficient and prone to error for each consumer of the combined 
research data to re-do the mapping independently, and the 
difficulty in harmonizing hinders the integration of the knowl- 
edge contained in their respective annotations into a broader 
understanding that spans genetic, molecular, cellular and psy- 
chological levels of description. As the underlying science pro- 
gresses, each of these ontologies will evolve in slightly different 
ways, rendering any mapping potentially out of date in the 
future. 

Rather than post hoc mapping efforts on a project-by-project 
basis, what is needed is a coordinated international effort towards 
ontology integration. Ontology integration involves the creation 
of bridging modules between ontologies that accurately reflect the 
shared understanding of the semantic relationships between the 
entities in the different ontologies. This is necessary in order to 
allow comparison and alignment (and thus maximally effective 
use) of empirical research data flowing from different sources and 
methodologies. Since the ontologies and the annotations (data) 
are separated, ontology-based integration does not change the way 
researchers in different areas work and in no way forces unwanted 
theoretical or methodological frameworks onto them. Rather, 
ontology-based integration flows from the use of a set of com- 
mon and integrated ontologies to annotate their data. We believe 
that with some coordinated effort it will be possible to create such 
common ontologies (in fact, we have already taken some steps in 
this direction, as described above) in a way that remains neutral 
as between different methodological approaches and fundamental 
assumptions. 

We believe is the way forward includes (a) continued use of 
existing ontologies for data annotation by primary data-generating 
researchers and deposition of those annotated data into shared 
repositories, alongside (b) ontology-based integration through the 
creation of shared semantic bridging modules between different 
ontologies and the increased convergence (as recommended by the 
OBO Foundry) on re-use of shared components between different 
ontologies, shared upper and mid-level ontologies, and shared 
metadata standards. 

We are currently building on existing active collaborations 
among ontology developers, and working towards creating a 
set of resources that the diverse interdisciplinary community of 
researchers will be able to stand behind. This will require a 
large-scale coordinated effort. However, in this era of increas- 
ingly data-driven science, as the focus turns to carefully retaining 
and making data available to drive reproducibility, we believe 
that there has never been a greater need, and also not a better 
opportunity. 

CONCLUSION 

We have described recent efforts in the development of cognitive 
ontologies and their use in annotation, integration and analysis of 
clinical and cognitive neuroscience data. A cross-cutting goal is to 
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identify classes of behavior and brain activity that are related to dif- 
ferent types of mental processes. This ambitious aim is consistent 
with recent calls for large-scale brain-mapping and support for 
translational neuroscience (e.g., Neuroimaging Informatics Tools 
and Resources Clearinghouse; Human Brain Project; the Brain 
Initiative - Devor etal., 2013), projects that are likely to benefit 
from the development and harmonization of cognitive ontolo- 
gies. There is a pressing need for coordinated efforts across related 
disciplines. The NIF plays a key role as an interconnecting hub and 
primary consumer for many of the ontology efforts described here. 
There is a real need to provide clear and community-negotiated 
definitions for core high-level entities within the domain, but for 
this effort to succeed, wide-scale community participation must 
be achieved. 

ACKNOWLEDGMENTS 

In July 2012, an international interdisciplinary group of 
scholars met in Graz, Austria to discuss the state of 
the art in cognitive ontologies across multiple disciplines 
(http://kr-med.org/icbofois2012/mfo/). The face-to-face meeting 
was preceded by a series of virtual seminars hosted by the NIF. 
The authors thank Prof. Stefan Schulz for support in running the 
workshop in Graz. They also thank all attendees of the workshop 
and pre-workshop seminar series for contributing to discussions. 
This work was partially supported by the Swiss Centre for Affective 
Sciences. Funding for the open access publication was provided by 
EMBL. 

REFERENCES 

Akil, H., Martone, M. E., and Van Essen, D. C. (2011). Challenges and opportunities 

in mining neuroscience data. Science 331, 708-712. doi: 10.1 126/science. 1199305 
Albus, J. S., Bekey, G. A., Holland, J. H., Kanwisher, N. G., Krichmar, J. L., Mishkin, 

M, etal. (2007). A proposal for a decade of the mind initiative. Science 317, 1321. 

doi: 10.1 126/science.317.5843. 1321b 
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M, etal. 

(2000). Gene ontology: tool for the unification of biology. Nat. Genet. 25, 25-29. 

doi: 10.1038/75556 

Bilder, R. M., Sabb, F. W., Parker, D. S., Kalar, D., Chu, W. W., Fox, J., etal. 
(2009). Cognitive ontologies for neuropsychiatric phenomics research. Cogn. 
Neuropsychiatry 14,419-450. doi: 10.1080/13546800902787180 

Burns, G. A. P. C, and Turner, J. A. (2013). Modeling functional mag- 
netic resonance imaging (fMRI) experimental variables in the ontology of 
experimental variables and values (OoEW). Neuroimage 15, 662-670. doi: 
10.1016/j.neuroimage.2013.05.024 

Ceusters, W., and Smith, B. (2010). Foundations for a realist ontology of mental 
disease. /. Biomed. Semantics 1, 10, 1-23. doi: 10.1186/2041-1480-1-10 

Cox, A. P., Jensen, M., Ruttenberg, A., Szigeti, K., and Diehl, A. D. (2013). "Mea- 
suring cognitive functions: hurdles in the development of the neuropsychological 
testing ontology," in Proceedings of the 4th International Conference on Biomedical 
Ontology 2013, Montreal, QC. 

Cryan, J. E, and Holmes, A. (2005). Model organisms: the ascent of mouse: advances 
in modelling human depression and anxiety. Nat. Rev. Drug Discov. 4, 775-790. 
doi: 10.1038/nrdl825 

Decety, J., and Cacioppo, J. (2010). Frontiers in human neuroscience: the 
golden triangle, and beyond. Perspect. Psychol. Sci. 5, 767-771. doi: 
10.1177/1745691610388780 

Devor, A., Bandettini, P. A., Boas, D. A., Bower, J. M., Buxton, R. B., Cohen, L. B., 
etal. (2013). The challenge of connecting the dots in the B.R.A.I.N. Neuron 80, 
270-274. doi: 10.1016/j.neuron.2013.09.008 

Fliessbach, K., Buerger, C, Trautner, P., Elger, C. E., and Weber, B. (2010). Differen- 
tial effects of semantic processing on memory encoding. Hum. Brain Mapp. 31, 
1653-1664. doi: 10.1002/hbm.20969 



Fox, P. T, and Friston, K. J. (2012). Distributed processing; distributed functions? 

Neuroimage 61, 407-426. doi: 10.1016/j.neuroimage.2011.12.051 
Frishkoff, G. (2012). "Mental functioning is neural functioning: towards a unified 

ontology of mind, brain, and behavior," in Proceedings of the Workshop on Mental 

Functioning Ontologies at the International Conference on Biomedical Ontology 

(ICBO'12), Graz. 

Frishkoff, G., Frank, R., and LePendu, P. (201 la). "Ontology-based analysis of event- 
related potentials," in Proceedings of the International Conference on Biomedical 
Ontology 2011, Buffalo, NY. 

Frishkoff, G., Sydes, J., Mueller, K., Frank, R., Curran, T, Connolly, J., 
etal. (2011b). Minimal Information for Neural Electromagnetic Ontologies 
(MINEMO): a standards -compliant method for analysis and integration of 
event-related potentials (ERP) data. Stand. Genomic Sci. 5, 211-223. doi: 
10.4056/sigs.2025347 

Frishkoff, G., LePendu, P., Frank, R., Liu, H., and Dou, D. (2009) Development 
of Neural Electromagnetic Ontologies (NEMO): Ontology-based tools for repre- 
sentation and integration of event-related brain potentials. In Proceedings of the 
International Conference on Biomedical Ontology 200% Buffalo, NY. 

Gardner, D., Akil, H., Ascoli, G. A., Bowden, D. M., Bug, W., Donohue, 
D. E., etal. (2008). The Neuroscience Information Framework: a data and 
knowledge environment for neuroscience. Neuroinformatics 6, 149-160. doi: 
10.1007/sl2021-008-9024-z 

Gkoutos, G. V., Schofield, P. N., and Hoehndorf, R. (2012). The Neurobehavior 
Ontology: an ontology for annotation and integration of behavior and behavioral 
phenotypes. Int. Rev. Neurobiol. 103, 69-87. doi: 10.1016/B978-0-12-388408- 
4.00004-6 

Hastings, J., le Novere, N., Ceusters, W., Mulligan, K., and Smith, B. (2012). 
"Wanting what we don't want to want: representing addiction in interoperable 
bio-ontologies," in Proceedings of the 3rd International Conference on Biomedical 
Ontology 2012, Graz, 897. 

Hastings, J., and Schulz, S. (2012). Ontologies for human behavior analysis and their 
application to clinical data. Int. Rev. Neurobiol. 103, 89-107. doi: 10.1016/B978- 
0-12-388408-4.00005-8 

Imam, F. T, Larson, S. D., Bandrowski, A., Grethe, J. S., Gupta, A., and 
Martone, M. E. (2012). Development and use of ontologies inside the neuro- 
science information framework: a practical approach. Front. Gene 3:111. doi: 
10.3389/fgene.2012.00111 

James, W. (1890). The Principles of Psychology, Vol. 2. New York, NY: Dover 
Publications, doi: 10.1037/10538-000 

Jensen, M., Cox, A. P., Chaudhry, N., Ng, M., Sule, D., Duncan, W., etal. (2013). 
The neurological disease ontology. /. Biomed. Semantics 4:42. doi: 10.1 186/2041- 
1480-4-42 

Jones, E., and Mendell, L. M. (1999). Assessing the decade of the brain. Science 284, 
739. doi: 10.1126/science.284.5415.739 

Kandel, E. R. (2008). Psychiatry, Psychoanalysis, and the New Biology of Mind. 
Arlington, VA: American Psychiatric Publishing. 

Laird, A. R., Lancaster, J. L., and Fox, P. T. (2005). BrainMap: the social evo- 
lution of a functional neuroimaging database. Neuroinformatics 3, 65-78. doi: 
10.1385/NI:3:1:065 

Laird, A. R., Lancaster, J. L., and Fox, P. T. (2009). Lost in localization? The focus is 
meta-analysis. Neuroimage AS, 18-20. doi: 10. 1016/j. neuroimage. 2009. 06. 047 

Mathur, P., and Guo, S. (2010). Use of zebrafish as a model to understand mecha- 
nisms of addiction and complex neurobehavioral phenotypes. Neurobiol. Dis. 40, 
66-72. doi: 10.1016/j.nbd.2010.05.016 

Mulligan, K., and Smith, B. (1986). A Relational Theory of the Act. Topoi5, 115-130. 
doi: 10.1007/BF00139225 

Noy, N. E, Shah, N. H., Whetzel, P. L., Dai, B., Dorf, M., Griffith, N., etal. (2009). 
Bio Portal: ontologies and integrated data resources at the click of a mouse. Nucleic 
AcidsRes. 37(Suppl. 2), W170-W173. doi: 10.1093/nar/gkp440 

Poldrack, R. A., Kittur, A., Kalar, D., Miller, E., Seppa, C, Gil, Y, etal. (2011). 
The Cognitive Atlas: towards a knowledge foundation for cognitive neuroscience. 
Front. Neuroinform. 5:17. doi: 10.3389/fninf.201 1.00017 

Poldrack, R. A., Mumford, J. A., Schonberg, T., Kalar, D., Barman, B., and 
Yarkoni, T. (2012). Discovering relations between mind, brain, and men- 
tal disorders using topic mapping. PLoS Comput. Biol. 8:el002707. doi: 
10.1371/journal.pcbi.l002707 

Raymond, E. S. (1999). The Cathedral and the Bazaar, ed. T. O'Reilly. Sebastopol, 
CA: O'Reilly 8t Associates, Inc. 



Frontiers in Neuroinformatics 



w w w.f ro n t i ersin.org 



June 2014 | Volume 8 | Article 62 | 6 



Hastings etal 



Perspectives on cognitive ontologies 



Rosse, C, and Mejino, J. L. (2003). A reference ontology for biomedical informat- 
ics: the foundational model of anatomy. /. Biomed. Inform. 36, 478-500. doi: 
10.1016/j.jbi.2003.1 1.007 

Smith, B., Ashburner, M., Rosse, C, Bard, C, Bug, W., Ceusters, W., etal. 
(2007). The OBO Foundry: coordinated evolution of ontologies to support 
biomedical data integration. Nat. Biotechnol. 25, 1251-1255. doi: 10.1038/ 
nbtl346 

Smith, B., and Ceusters, W. (2010). Ontological realism: a methodology for 

coordinated evolution of scientific ontologies. Appl. Ontol. 5, 139-188. 
Smith, B., and Grenon, P. (2004). The cornucopia of formal-ontological relations. 

Dialectica 58, 279-296. doi: 10.1111/j.l746-8361.2004.tb00305.x 
Turner, J. A., and Laird, A. R. (2012). The cognitive paradigm ontology: design and 

application. Neuroinformatics 10, 57-66. doi: 10.1007/sl2021-011-9126-x 
Weyandt, L. L. (2006). The Physiological Bases of Cognitive and Behavioral Disorders. 

Routledge: Taylor and Francis. 
Wu, Y., and Luo, Y. (2005). Transgenic C. elegans as a model in Alzheimer's research. 

Curr. Alzheimer Res. 2, 37-45. doi: 10.2174/1567205052772768 



Conflict of Interest Statement: The authors declare that the research was conducted 
in the absence of any commercial or financial relationships that could be construed 
as a potential conflict of interest. 

Received: 25 November 2013; accepted: 02 June 2014; published online: 20 June 2014. 
Citation: Hastings J, Frishkoff GA, Smith B, Jensen M, Poldrack RA, Lomax J, 
Bandrowski A, Imam F, Turner JA and Martone MF (2014) Interdisciplinary perspec- 
tives on the development, integration, and application of cognitive ontologies. Front. 
Neuroinform. 8:62. doi: 10.3389/fninf.2014.00062 
This article was submitted to the Journal Frontiers in Neuroinformatics. 
Copyright © 2014 Hastings, Frishkoff, Smith, Jensen, Poldrack, Lomax, Bandrowski, 
Imam, Turner and Martone. This is an open-access article distributed under the terms 
of the Creative Commons Attribution License (CC BY). The use, distribution or repro- 
duction in other forums is permitted, provided the original author(s) or licensor are 
credited and that the original publication in this journal is cited, in accordance with 
accepted academic practice. No use, distribution or reproduction is permitted which 
does not comply with these terms. 



Frontiers in Neuroinformatics 



www.frontiersin.org 



June 2014 | Volume 8 I Article 62 | 7 



